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(57) ABSTRACT 

A method and apparatus thai enables a user to store event 
information while watching a television broadcast is dis- 
closed. The event information is transmitted to a server, 
preferably via the Internet. Based on the event information, 
the server determines which program the user was watching 
when the event information was stored, and the time within 
that program. Based on the determined program and time, 
the server determines an assortment of goods and services 
that were displayed on the user's television when the event 
information was stored. This assortment is presented to the 
user, preferably via the Internet. 
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INTEGRATED TELEVISION AND INTERNET 
INFORMATION SYSTEM 

BACKGROUND OF THE INVENTION 

This invention relates to the field of facilitating commerce 
by providing access to information about goods and services 
that are displayed in television broadcasts. 

Using television commercials to promote products is a 
widespread practice. Typically, viewers watch certain pro- 
grams on television because they are interested in those 
programs. Television stations capitalize on this situation by 
broadcasting commercials interspersed with the program 
itself. These commercials typically try to convince the 
viewer to buy a particular product. As used herein, the term 
"product" refers to both goods and services. 

Interspersing commercials within television programs, 
however, has some significant drawbacks. For example, 
from the perspective of the viewer, television commercials 
are often perceived as annoying interruptions to ihe program 
that the viewer wishes to watch. Even when a viewer is 
interested in a particular product in a commercial, the viewer 
may prefer to find out about it after he has finished watching 
the program. This can not be accomplished with conven- 
tional television broadcasts. 

Another drawback, from the viewer's perspective, is that 
the viewer has no control over the subject matter of the 
commercials thai be will see, since the content of traditional 
commercials is determined solely by the advertisers. As a 
result, many viewers may not be interested in the commer- 
cials that they see. 

In contrast, since the viewer selects the program that he is 
watching, the viewer is presumably interested in the con- 
tents of that program. During a conventional broadcast of the 
program itself, however, the viewer has no way to obtain 
information about any products that he sees in the program 
(e.g., acertainjacketbeingwornby an actor in a TV sitcom). 

Traditional commercials also have drawbacks from the 
broadcaster's perspective. First, because commercials only 
take up a relatively small fraction of the total time of a 
broadcast, the amount of air time that broadcasters can sell 
is limited. And if a broadcaster attempts to overcome this 
limitation by increasing the amount of commercials, the 
broadcaster risks losing it's viewers, because they may 
switch to other stations, only to return once the commercial 
is over. Hence, no benefit is obtained for the advertiser, the 
broadcaster, and the consumer. 

The Internet is another conventional forum in which 
products are advertised. These advertisements typically take 
the form of advertisements displayed on a website and 
generally appear on the screen simultaneously with the 
desired information, in a distinct section of the display. 
Unlike traditional television commercials, such advertise- 
ments do not preclude the user from viewing other infor- 
mation. If the user is interested in the Internet advertisement, 
the user can click on the advertisement and proceed accord- 
ingly. If on the other hand, the user is not interested, the user 
simply ignores the advertisement and continues obtaining 
the information in which he is interested. Internet advertise- 
ments are therefore less intrusive than television commer- 
cials. Internet advertisements also provide an added benefit, 
in that, once an interested customer has been found, a sale 
transaction can be facilitated on the spot. 

Despite these advantages, however, product commercial- 
ization using the Internet can only take place when the 
relevant users are actually logged on to the Internet. Even 
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among families with access to the Internet, however, the 
total amount of time spent logged on to the Internet in (his 
country is dwarfed by the total amount of time spent 
watching television. As a result, the marketing power of 
S Internet advertisements is still relatively insignificant in 
comparison to the marketing power of television commer- 
cials. 

A commercialization system that does not suffer from the 
drawbacks described above would be desirable to both 

to viewers and broadcasters. Until now, however, no such 
systems have been implemented. 

In the past, attempts have been made to combine televi- 
sion and Internet commercialization. The simplest of these 
combinations is displaying a URL (uniform resource 

15 locator), or "address," of a website on the television during 
a traditional television commercial. This system is problem- 
atic for both the viewer and the advertiser, because the 
viewer may not have a pen handy to write the URL down. 
And even in cases where the user does write the URL down, 

20 the paper may not be handy the next time the user logs on 
to the Internet. Either of these scenarios would result in a lost 
opportunity to promote a sale, and a lost opportunity for a 
consumer to obtain a product in which he is interested. 

25 Another existing combined television/Internet system is 
WebTV. WebTV allows its users to view a website on a 
traditional television display. Recent innovations have even 
allowed a website to be accessed without interrupting the 
television signal, using a picture in picture format. But in 

30 WebTV, the Internet and television-viewing sections are 
largely independent, and there is no interaction between the 
URL accessed and the television program being viewed. 
Finding information about products that appear within a 
show must be accomplished using traditional search meth- 

35 ods. 

Yet another combined television/Internet system is 
described in U.S. Pat. No. 5,778,181. The '181 patent 
describes transmitting URLs during the vertical blanking 
interval of a television broadcasting signal. These URLs are 

40 extracted at the viewer's home, and the associated web 
pages are fetched (via the Internet) while the viewer is still 
watching his television. In the '181 patent, however, the 
viewer is merely fed two streams of information, with one 
displayed on the television, and the other displayed on a 

45 computer monitor. As a result, many of the disadvantages of 
traditional television viewing remain. In addition, the user 
must divert his attention away from the program that he is 
watching in order to obtain the information being displayed 
on the Internet. 

50 SUMMARY OF THE INVENTION 

The present invention advantageously overcomes many 
of the aforementioned disadvantages and provides an inte- 
grated television/Internet commercialization system. This 

55 system enables a user to watch television in a traditional 
manner, but also provides the user with an opportunity to 
indicate interest in things that are being displayed on the 
television. This indication could be made, for example, by 
pressing a button on a customized handheld remote control 

60 The system stores these indications. After the user has 
finished watching the program (or at such other time, as the 
user may desire), these indications are transferred to a 
remote server, which presents to the user information about 
the products that were being displayed at the time the user 

65 made each indication. 

According to one aspect of Ihe present invention, a 
method of commercializing products that are present in a 
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television broadcast of a program is provided. The method 
includes the steps of inputting product information that 
identifies a plurality of products and associated times pres- 
ence in the program, inputting skew information that iden- 
tifies a correspondence between an actual time of broadcast 
and a relative time within the program, and inputting a time 
marker that identifies a time selected by a user. The method 
also includes the steps of identifying a specific portion of the 
program that was being broadcast at the time selected by the 
user based on the time identified by the time marker and the 
inputted skew information, determining an assortment of 
products that were present in the television broadcast at the 
selected time, and presenting the determined assortment of 
products to the user. 

According to another aspect of the present invention, a 
method of commercializing products that are present in 
multiple simultaneous television broadcasts of programs is 
provided. The method includes the steps of inputting, for 
each of the programs, product information that identifies 
products which are present in the respective program and 
associated times of presence, inputting skew information 
that identifies a correspondence between an actual time of 
broadcast and a relative time within the respective program, 
and inputting a marker that identifies a selected time and 
channel. The method also includes the steps of identifying a 
specific portion of the program that was being watched by 
the user based on the marker and the inputted skew 
information, determining an assortment of products thai 
were present in that program at the selected time, and 
presenting the determined assortment of products to the user. 

According to another aspect of the present invention, a 
method of commercializing products that are present in 
multiple simultaneous television broadcasts of programs is 
provided. The method includes the steps of inputting, for 
each of the programs, broadcast and product information 
that identifies products which are present in the broadcasts 
and times of presence for each of the products, and inputting 
a marker that identifies a time selected by a user and a 
channel being watched. The method also includes the steps 
of determining an assortment of products that were present 
in the program that was being watched al the selected time 
based on the time and channel identified in the marker and 
the inputted broadcast and product information, and present- 
ing the determined assortment of products to the user. 

According to yet another aspect of the present invention, 
a method of marking an event is provided. The method 
includes the steps of keeping track of time, waiting for an 
event indication from a user, storing the tracked time in 
response to receipt of the event indication, waiting for a 
transmit command, and transmitting the stored time to an 
external device. The external device then determines the 
products that were present in the television broadcast at the 
time the event indication occurred based on the lime trans- 
mitted in the transmitting step. 

According to still another aspect of the present invention, 
a method of marking an event is provided. The method 
includes the steps of keeping track of time, keeping track of 
which channel is being displayed on a television, waiting for 
an event indication from a user, storing the tracked time and 
the tracked channel in response to receipt of the event 
indication, waiting for a transmit command, and transmitting 
the stored time and the stored channel to an external device. 

The above and other features and advantages of the 
present invention will be apparent from the following 
detailed description of preferred embodiments. The detailed 
description is to be read in connection with the accompa- 
nying drawings, in which: 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a system overview block diagram of a preferred 
embodiment of the present invention. 

FIG. 2 is a block diagram of a handheld remote of the 
preferred embodiment. 

FIG. 3 is an example of a user's TV setup which can be 
used with the present invention. 

FIG. 4 is a table specifying the user's TV setup. 

FIG. 5 is a table specifying the channel setting for each 
tuner device. 

FIG. 6 is a table of internal variables stored in the 
handheld remote. 

FIG. 7 is a table of time stamps, which is also stored in the 
handheld remote. 

FIG. 8 is a flowchart depicting the operation of the 
handheld remote. 

FIG. 9 is a flowchart depicting processing of universal 
remote keys in the handheld remote. 

FIG. 10 is a block diagram of a home computer setup used 
with the preferred embodiment. 

FIG. 11 is a flowchart depicting the transmission of time 
stamps from the handheld remote. 

FIG. 12 is a flowchart depicting the reception of time 
stamps by the home computer. 

FIG. 13 is a flowchart depicting the processing of time 
stamps by the home computer. 

FIG. 14 illustrates mapping broadcast time into program 
time using time lines. 

FIG. 15 is a table indicating which channel is used by each 
network for each service provider. 

FIG. 16Ais a table that stores information relating to the 
programs supported by the system. 

FIG. 16B is a table reporting the broadcasting of a 
program for a particular network. 

FIG. 17 is a flowchart depicting processing that occurs in 
the central server. 

FIG. 18 is a flowchart depicting the creation of the 
Product Display Table. 

FIG. 19 is a table indicating when each product appears 
and disappears in a given program. 

FIG. 20 is a table of product information. 

FIG. 21 is a flowchart depicting the conversion of time 
stamps to products. 

FIG. 22 depicts an alternative embodiment that is imple- 
mented in an application space on a computer. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

FIG. 1 is a block diagram of a preferred embodiment in 
accordance with the present invention. In this embodiment, 
television programs are broadcast from the broadcaster 104 
(e.g., a television station) to the user's home in any con- 
ventional manner, including for example, broadcasts via a 
cable service 108. Alternatively, other types of broadcasts 
can be used, including, for example, satellite and ground 
based antennas transmissions (not shown). Traditional tele- 
vision commercials may be interspersed with the programs 
and broadcast over the same medium in a conventional 
manner. 

The broadcast signals are distributed within the users 's 
home in any conventional manner. In the illustrated setup, 
the cable signal is provided directly to the television 102, 
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and also indirectly, via the cable box 103. Notably, no 
modifications to the conventional broadcasting equipment 
are required, and no special equipment is required by 
preexisting customers who are not using the present inven- 
tion. 

Preferably, the system is implemented using a customized 
handheld remote control unit 105 which performs all con- 
ventional universal remote functions. These functions arc 
well known and include, for example, selecting a signal 
source, changing the channel on the currently selected signal 
source, and changing the volume on the TV set. Each of 
these functions is preferably implemented using conven- 
tional buttons on the handheld remote. 

In addition to these conventional buttons, the handheld 
remote unit 105 of the present invention preferably includes 
two additional buttons: an event button and a transfer button. 

From the user's perspective, the experience of watching 
television 102 starts out the same as when he uses a 
conventional universal remote. For example, he can use the 
remote 105 to change channels or adjust the volume at any 
time. Whenever the user 101 sees something on his televi- 
sion 102 in which he is interested, the user presses the event 
button on the handheld remote 105, and continues to watch 
the program. Thus, the only change (as compares to tradi- 
tional television watching) is the occasional pressing of the 
event button, which is a minimal interruption. 

Each time the user 101 presses the event button, the 
remote 105 stores event information. Preferably, this event 
information includes the time that the event button was 
pressed, as well as the signal source and channel being 
watched at that time. This is described below in greater 
detail. 

Once the user 101 finishes watching television 102, he 
takes the handheld remote 105 over to bis home computer 
106. Preferably, both the remote and the computer are 
equipped with displays and IR (infrared) transceivers. The 
user then points the remote's IR transceiver at the IR 
transceiver on the computer, and presses the transfer button 
on the remote to establish a communication session with the 
home computer. Set-up of this communication session may 
be facilitated by appropriate prompts to the user 101 on the 
displays on the remote 105 and the home computer 106. 
During the communication session, information pertaining 
to each press of the event button on the remote is transferred 
into the home computer. Alternatively, in place of the IR 
communication described herein, other types of communi- 
cation links including, for example, radio frequency and 
X-10 communication, may be used. 

After the event information has been transferred from the 
handheld remote 105 into the home computer 106, the home 
computer 106 establishes a connection with the central 
server 107 via the Internet, and sends the event information 
to the server. 

Meanwhile, the server has previously inputted product 
information that identifies products which are present in 
various programs, and times of presence within the pro- 
grams for each of the products. This information can come, 
for example, from a database created by the producers of a 
program. Alternatively, this information could be inputted 
by a person watching the program who has noted the 
products in the program and the portion of the program in 
which those products appear. Numerous other scenarios can 
be readily envisioned. 

In addition, the server has previously inputted skew 
information, described in greater detail below, which indi- 
cates when each segment of a program is actually broadcast. 



is 



25 



The server uses this skew information to map "broadcast 
time" (which is an actual time, such as Jan. 2, 1999, 8:05:02 
PM) into "program time" (which is a relative time within the 
uncut program, such as 56 minutes and 22 seconds from the 
5 start of a particular program). Preferably, the skew informa- 
tion will arrive from a cooperating broadcaster thai sends the 
skew information to the server while the program is being 
broadcast. 

The server 107 then determines an assortment of products 
10 that were displayed on the user's television 102 when the 
user pressed the event button on the remote 105, for each 
press of the event button. This is accomplished by referenc- 
ing a previously inputted product data base that indicates 
which products appear in the program being watched, and 
the times that they appear (measured in program time). 

The central server 107 then sends information about this 
assortment of products back to the home computer 106 via 
the Internet. This can be accomplished, for example, by 
sending a web page or database to the home computer 106. 
20 The product assortment Is then presented to the user 101 on 
the display of home computer 106. The product assortment 
can be presented to the user in any number of ways. For 
example, a set of windows can be used, with one window 
representing each product in the assortment. Alternatively, a 
still image or a video clip of the selected moment of the 
program may be displayed. Numerous other alternatives 
presentation approaches can be readily envisioned. 

This arrangement is advantageous to broadcasters 
because it expands commercialization into the time that the 
television program itself is being broadcast. It can even be 
used to commercialize products that appear during the 
commercials themselves (such as the shoes being worn by 
the spokesperson in a commercial for a car). 

It is also advantageous to television viewers because it 
enables them to obtain information on and purchase prod- 
ucts which attracted their interest. 

FIG. 2 is block diagram of the handheld remote unit. 
Preferably, it includes a processor 201 that is connected to 
RAM (Random Access Memory) 202, ROM (Read Only 
Memory) 203, and an I/O adapter 204 via a system bus. 
These components and their interconnections are well- 
known, as evidenced by the large number of universal 
remote controls available today. Most preferably, all of these 
components are integrated into a single integrated circuit. 

Of course, various equivalents for each of these compo- 
nents may be substituted for the respective component. 
Examples of such substitutions include using hardwired 
logic in place of a processor running a program out of ROM, 
50 or using a RAM with a battery backup in place of the ROM. 
Numerous other modifications will be apparent to persons 
skilled in the relevant art. 

The I/O adapter 204 enables the processor 201 to deter- 
mine which keys on the keypad 205 are being pressed, in any 
conventional manner. In addition, the I/O adapter 204 
enables the processor 201 to communicate with an IR 
transceiver 207. The Vishay Telefunken TFDT5500 is an 
example of a suitable IR transceiver 207. The I/O adapter 
204 also enables the processor 201 to write to the liquid 
crystal display 206, also in any conventional manner. 

The handheld remote performs three distinct functioas: 
control of the devices in the user's home (e.g., the television 
102 and the cable box 103 shown in FIG. 1), recording of 
events, and communication with the home computer 106 
(also shown in FIG. 1). 

The control mode is used to send appropriate IR control 
codes to various devices in the users home, as with conven- 
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tional universal remote controls. These control codes could 
include, for example, changing the volume on the TV set, or 
changing the channel on the cable box. Communicating with 
such devices via IR commands is well-known, as exempli- 
fied by commercially available universal remote control 
devices. In this mode, the IR transceiver 207 operates as a 
transmit-only device, and the processor 201 controls the IR 
transceiver 207 by sending signals to the I/O adapter 204. 

The event recording mode is used by a television viewer 
to indicate that he is interested in a product that appears on 
the television. Preferably, this is implemented using a dedi- 
cated button located on the keypad 205, which the viewer 
presses to indicate that he is interested in a product that 
appears on the television. This button is referred to herein as 
the "event" button, which is not implemented on conven- 
tional universal remotes. Preferably, when a key-press for 
the event button is detected, the processor 201 in the remote 
stores appropriate information in memory in tables set up in 
RAM 202. These tables are explained in greater detail 
below. 

In the communication mode, the remote sends the event 
information, which was stored in response to presses of the 
event button, to the computer 106. Preferably, this is imple- 
mented using a second dedicated button located on the 
keypad 205, which the viewer presses when he wants to 
transfer information from the remote 105 into the computer 
106 (for subsequent uploading to the server 107). This 
button is referred to herein as the "transmit" button, which 
is also not implemented on conventional universal remotes. 
When a key-press for the transfer button is detected, the 
processor 201 initiates communication with the home com- 
puter 106, via the IR transceiver 207. It is envisioned that 
communication with the user's home computer will usually 
take place after the user has finished watching television, 
although this is not mandatory. 

In this mode, the IR transceiver 207 in the handheld 
remote 105 is used to communicate with the home computer 
106. While it is only necessary to transfer information in one 
direction (from the remote to the computer), the system is 
preferably implemented using two-way communication, to 
facilitate handshaking and to ensure that the data is trans- 
ferred properly. Communication between home computers 
and other devices using IR transceivers is also well known, 
as evidenced by conventional IR technology based cordless 
computer keyboards, communications between handheld 
personal organizers, and the IRDA standard for such com- 
munications. 

A single IR transceiver may be used for both the control 
mode and the communication mode, as explained above. 
Alternatively, the IR transceiver (e.g., an IRDA compliant 
transceiver) may be used only for communication with the 
computer. An independent IR transmitter would then be used 
for controlling the entertainment devices. 

In the embodiment described above, the device control 
functions, the event button, and the transmit button arc 
implemented on a handheld remote. Alternative embodi- 
ments can be readily envisioned, including, for example, 
implementing all these functions on a home computer (or 
combined televisional home computer), and actuating them 
.by pressing keys on the keyboard or with mouse clicks. 

FIG. 3 depicts one of many possible cable television 
setups that could appear in a user's home, which will be used 
as an example to explain the present invention. Of course, 
the invention may be practiced with other setups, including, 
for example, setups that receive broadcasts from a land 
based antenna broadcasting VHF signals, a satellite, a Local 
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Area Network, or the Internet. In the depicted configuration, 
a cable service 108 is hooked up directly to the "cable" input 
of television 102. The cable service is also routed to a cable 
set top box 103, which in turn sends a signal to the "aux" 

5 input of the television set 102. With this setup, the user is 
free to select either the cable input (using the tuner in the 
television) or the aux input (using the tuner in the cable box) 
for display on the television 102. 

Preferably, the handheld remote can control selection of 

10 the signal source that is displayed on the television 102. In 
addition, the handheld remote can control the channel to 
which the television is set, as well as the channel for any 
other tuner device (e.g. the cable box 103) in the system. 
This is preferably accomplished by issuing IR remote com- 

1S mands. In order to control these devices, the handheld 
remote must be initialized to know which brands of equip- 
ment are installed in the user's home, so that it can send out 
the appropriate IR commands for that equipment. For 
example, if the television set 102 was made by Sony, the 

20 remote must know this in order to issue the appropriate IR 
commands to control the television 102. 

Initialization of the remote is preferably accomplished via 
IR commands that are transmitted to the remote from the 
home computer via IR transceivers. Alternatively, other 

25 means for initializing the remote may be used, including, but 
not limited to, providing a printed table that lists codes for 
each manufacturer, and instructions asking the user to enter 
these codes using the buttons on the handheld remote. 

30 The initialization information that specifies the user's 
system setup is preferably stored in a file on the home 
computer in a service and device configuration table 250 
(hereinafter the SDC table, shown in FIG. 4). Each row of 
this table relates to one device that can display information 

35 on the user's television. The table includes information 
specifying an input of the television, the device used to tune 
to a particular frequency, the manufacturer of that device, the 
service name, and a service ID. Preferably, this table is 
stored in the home computer and created using a program 

^ that lets the user enter information defining his video equip- 
ment setup. 

In the illustrated example, the second row of the SDC 
table 250 indicates that to display a station arriving via the 
cable box, the television is set to select the aux input, that the 

45 cable service providing signals to the cable box is Manhattan 
Cable, that the service ID for Manhattan Cable is 3888, and 
that the cable box is made by General Instrument. The third 
row of the SDC table contains corresponding information 
for signals arriving directly at the television's cable input, 

50 without passing through the cable box. In that case, the 
station is selected using the built-in tuner on the Sony 
television. In this example, both rows have the same service 
ID, because both the cable box and the cable input of the 
television are fed by the same cable service. When a single 

55 user receives service from multiple providers (e.g., cable and 
satellite), the SDC table would list a different service ID for 
each provider. 

FIG. 5 shows an example of a tuner device current state 
table 260, which is used by the remote to keep track of the 

60 channel to which each tuner device is tuned. Preferably, this 
table is also stored in RAM in the remote. In the illustrated 
example, this table stores information indicating that the 
television tuner is set to channel 5, and that the cable box is 
set to channel 4. Because the handheld remote is used as a 

65 universal remote to control all the television related devices 
in the user's home, it is simple for the handheld remote to 
keep track of the particular channel to which each of the 
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tuner devices is tuned. This can be accomplished, for 
example, by updating an internal register after each channel- 
changing command is issued to any device. 

FIG. 6 illustrates a handheld remote internal variables 
table 270. This table includes fields for the date, the time, 5 
and the current display device. Keeping track of the date and 
time may be accomplished by any number of conventional 
methods including, for example, using a built-in timekeep- 
ing feature of the processor in the handheld remote unit, or 
software that runs on that processor. The table 270 also 10 
includes a field, preferably stored in RAM, which tracks the 
current display device. In the illustrated example, this field 
would be set to a first value when the television's cable input 
is selected, and another value when the television's aux 
input is selected. The handheld remote keeps this field 15 
current by updating it after issuing any command which 
changes the signal source that is being displayed on the 
television. Preferably, when two or more images are dis- 
played on the television, as with picture-in-picture 
televisions, the largest display will be tracked. 20 

When the event button on the handheld remote is pressed, 
information from the tuner device current state table 260 and 
the handheld remote internal variables table 270 are 
extracted and stored in a time stamp table 280, shown in 
FIG. 7. Each time the event button is pressed, an additional 25 
entry is added to the time stamp table 280. These entries are 
referred to herein as "time stamps" or "markers". The 
example illustrated in FIG. 7 includes four such time stamps. 
The number of time stamps that can be stored in the time 
stamp table 280 may be limited at a preset number, or may 30 
be determined by the amount of available RAM available in 
the handheld remote. Preferably, time stamps are erased 
from the time stamp table when they are transferred to the 
home computer. 

FIG. 8 illustrates the functions performed in the handheld 
remote, which are preferably performed in software running 
on the processor within the remote. In step S300, the 
program waits for a key to be pressed. This can be accom- 
plished in any number of ways including, for example, 4Q 
receiving an interrupt from a keyboard interface or by 
strobing an input port until a key press has been detected. 
Once a key press has been detected, processing proceeds to 
step S310. 

In step S310, the pressed key is tested to determine 45 
whether it is universal remote key, such as a volume control 
key, or a channel changing key. If the pressed key is a 
universal remote key, then the key is processed in step S311. 
If the pressed key is not a universal remote key, then 
processing continues to step S320, where a test is performed 50 
to determine whether the pressed key is the event button 
(referred to in FIG. 8 as the record time stamp key). 

If the pressed key is the event button, it is processed in 
step S321, where the remote stores the date, time, channel 
and current device in the time stamp table 280, thereby 55 
creating an entry in that table. In the example depicted in 
FIG. 7, the second entry indicates that the event button was 
pressed while the user was watching channel 5 from the 
cable box on Nov. 10, 1998 at 8:26 PM. Alternatively, in 
embodiments where only one station is supported by the go 
system, the channel and current device information can be 
omitted from the time stamp table, because those fields 
would always contain the same information. Returning now 
to FIG. 8, the time stamp data for the event is displayed on 
the LCD of the handheld remote. 65 

If, in step S320, the pressed key is not the event button, 
processing continues to step S330 where a test is performed 
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to determine whether the pressed key is the transfer com- 
mand (referred to in FIG. 8 as the download time stamps 
key). If it is a transfer command, the time stamps are 
transmitted to a computer receiver via the IR transceiver in 
step S331. If the pressed key is not a transfer command, 
processing continues in step S340. 

Step 340 is implemented only in embodiments that sup- 
port browsing directly from a handheld remote, which is an 
optional feature. Dedicated keys may be added to the remote 
for this purpose. In step S340, the pressed key is tested to 
determine whether it is a browser key. If it is, processing of 
the browser key occurs in step S341, which preferably 
forwards the browser key directly to the computer. When 
browsing from the remote is not implemented, browsing can 
be accomplished in any traditional manner, such as using a 
mouse attached to the home computer. 

Finally, after the pressed key is processed, the program 
returns to step S300 to wait for the next key press. 

FIG. 9 describes the processing of universal remote keys. 
Processing begins in step S400. In step S410, if the pressed 
key is a power key for a tuner device (i.e., a key which turns 
on a tuner), then that key is processed in step S41.1. In step 
S411, the remote issues an IR command to turn on the 
device, followed by an IR control command which changes 
the device's channel to the channel that is already stored in 
the handheld remote's internal register. This channel setting 
is obtained from the tuner device current state table 260 
(shown in FIG. 5). This ensures that the variables in the 
handheld remote match the actual settings of the devices, 
forcing the devices to a known state. 

If the pressed key is not a power key, processing continues 
in step S420, where a test is performed to determine whether 
the pressed key is a channel change key. If it is a channel 
change key, the pressed key is processed in step S421. In 
step S421, the remote issues an IR command to change (he 
channel on the currently selected device. It also stores the 
new channel setting in the tuner device current slate table 
260 (shown in FIG. 5). 

If the pressed key is not a channel change key, then 
processing continues in step S430 where a test is performed 
to determine whether the pressed key is a display device 
change key. If the pressed key is a display device change 
key, it is processed in step S431. In step S431, the handheld 
remote issues an IR command to change the display device 
being displayed on the television. The new current display 
setting is also stored in the handheld remote internal vari- 
ables table 270 (shown in FIG. 6). 

If the pressed key is not a display device change key, then 
processing continues to step S440 where the handheld 
remote issues an IR command corresponding to the key 
pressed. Keys that are processed in this step include all keys 
that are not power, channel change, or display change 
command keys (e.g., a volume change command). Finally, 
this subroutine ends in step 450. 

FIG. 10 depicts an example of a home computer setup 
suitable for use with the present invention. The home 
computer 502 is connected to appropriate interface circuitry 
501, either via an internal bus, or via a connector and a cable. 
This interface circuitry 501 enables the computer 502 to 
communicate with the IR transceiver 500. The connections 
and communication between the computer 502 and the IR 
transceiver 500 can be implemented in any conventional 
manner. One example of a suitable device for performing 
this function is the Actisys IR220L. 

To establish communication with the home computer 502, 
the user points his handheld remote at the IR transceiver 500 
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connected lo ihe computer 502. The user then presses the 
transfer button on his handheld remote to transfer the lime 
stamps stored in the remote to the computer. These lime 
stamps are received by the computer 502 via the interface 
circuitry 501. 5 

Although the transfer of time stamps from the remote to 
the home computer described above only involves data 
flowing in one direction (i.e., from the remote to the 
computer), two-way communication may also be used to 
implement handshaking or to improve the reliability of the 10 
data transmission, in any conventional manner. Two-way 
communication is also useful to initialize (or re-initialize) 
the remote as to which brands of equipment are installed in 
the user's home, as explained above. Optionally, two-way 
communication may be used to send messages to the user via J5 
the LCD display on the remote. 

FIG. 11 shows the processing that occurs in the, remote 
when data is sent from the remote to the computer using 
two-way communication. First, in step S610, the time 
stamps stored in the time stamp table 280 (shown in FIG. 7) 2 o 
are encoded as binary pulses. Then, in step S620, the user is 
prompted to point the handheld remote at the computer's IR 
transceiver. This prompt will appear on the LCD display of 
the handheld remote in embodiments that have such a 
display. Next, in step S630, the binary pulses are transmitted 25 
to the computer via the IR transceiver. In step S640, the 
remote waits for acknowledgment of the time stamps that 
were transmitted to the computer. In step S650, a test is 
performed to determine whether the transmitted data was 
received correctly by the computer, preferably based on the 30 
acknowledgment received in step S640. If the transmitted 
time stamps were not received correctly by the computer, 
processing returns to step S620. If the data was received 
correctly, processing of this subroutine ends. 

FIG. 12 depicts the process that runs in the computer to 35 
communicate with the handheld remote. Processing begins 
in step S700 where the computer waits for an IR commu- 
nication signal to arrive from the handheld remote. When the 
communication signal arrives, a test is performed in step 
S710 to determine whether the data was received correctly. 40 
If the data was not received correctly, the computer prompts 
the user in step S711, using a suitable display or audio 
message, to point the remote at the computer IR transceiver. 
If it is determined in step S710 that the data was received 
correctly, processing continues in step S720 to determine 45 
whether the received data represents time stamps. If the 
received data represents time stamps, the time stamps are 
processed in step S721. 

If the received data is not a time stamp, the received data 
is tested to determine whether it is a browser key in step 50 
S730 (in embodiments that support browsing from the 
remote). If it is a browser key, it is processed in step S731. 
This processing would include forwarding the browser key 
to a web browser running on the home computer, which then 
communicates with the server via the Internet in a conven- 55 
tional manner. 

FIG. 13 depicts bow the time stamps are received and 
processed by the computer. After the time stamps are 
received in step S810, the received time stamps are tested in 
step S820 to determine whether the data was received 60 
correctly. If the data was nol received correctly, the com- 
puter prompts the user, in step S821, to point the handheld 
remote at the computer IR transceiver. If the data was 
received correctly, the time stamps that were encoded in the 
handheld remote are decoded in step S830. Those time 65 
stamps are then stored in a time stamp table in the computer 
in step S840. 
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Next, in step S850, the computer will request permission 
from the user to begin processing an order and test for 
receipt of this permission in step S860. If permission is not 
granted (which might occur, for example, if the phone line 
used for Internet access is being used by another person), the 
subroutine ends in step S870. If permission is granted, order 
processing is set up in step S861, in which case the computer 
will establish a connection to the order processing server site 
via the Internet in any conventional manner, and forward the 
time stamps and the SDC table information to the server. 
Preferably, the connection to the server site uses the hyper- 
text transfer protocol (HTTP). Once a connection has been 
suitably established, the computer encrypts the time stamp 
information and the SDC table and, optionally, information 
related to the user (including, e.g., account information). 
This encrypted information is transmitted to the server via 
the Internet, also in a conventional manner. Communication 
with a server then continues as with any other website. 

Once the server receives the time stamp, the server must 
compensate for all time shifts that were introduced when the 
program was broadcast. This is accomplished by mapping 
each second of the broadcast, which occurs in real time, into 
a second of the original uncut program. This process com- 
pensates for lime shift and skews caused by, for example, 
delays in starting a program, and commercials that are 
interspersed with the program. 

FIG. 14 shows an example of this mapping. The "program 
time" time line 900 shows the time with respect to the 
original program. The "broadcast time" time line 901 shows 
the actual times that each segment of the program was 
broadcast. Ordinarily, each moment of program time will 
map into only one corresponding moment of broadcast time. 
In the illustrated example, minutes 20 through 32 of the 
program were broadcasted from 8:22 PM to 8:34 PM on Jan. 
1, 1999. After this 12 minute segment, 3 minutes of com- 
mercials were broadcasted from 8:34 PM to 8:37 PM. Then, 
the next 8 minute segment of the program was broadcasted 
from 8:37 PM to 8:45 PM. Next, a 4 minute commercial was 
broadcasted from 8:45 PM to 8:49 PM. Minutes 40-44 of 
the program were never properly broadcast due to technical 
difficulties, which is indicated in the broadcast time line 
from 8:49-8:53 PM. The next section of the program 
(beginning at minute 44), was broadcasted starting at 8:53 
PM. 

The process of mapping broadcast time to program time 
is referred to as skew compensation. This skew compensa- 
tion process can be implemented for programs of any length, 
including, for example, a half-hour sitcom episode, a two 
hour movie, (which, when commercials are added, might 
take 2.5 hours to broadcast), or even a three hour movie, 
broadcasted in two or more installments on different days. In 
the latter case, the skew compensation process maps two or 
more broadcast time lines (one for each day), onto the 
original three hours of program time. The server uses this 
map to convert the real time information from each time 
stamp into a corresponding program time, which is the time 
in the program that was being broadcast when the event 
button was pressed. The server can use the determined 
program time to index into a product database that identifies 
the products that appear at each time in the program, and 
thereby determine which products were being displayed at 
the time the event button was pressed. This process of skew 
compensation is described in greater detail below. 

In addition to obtaining the information contained in the 
lime stamps, the server also must obtain information about 
the television setup in the user's home. Preferably, this is 
accomplished by obtaining a copy of the service and device 
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configuration table 250 (shown in FIG. 4, hereinafter "the 
SDC table"). This is preferably accomplished by transfer- 
ring a copy of the SDC table from the home computer to the 
server each time a set of lime stamps is transferred from the 
home computer to the server. 

Optionally, customer information for each user may also 
be stored in the home computer, and transferred to the server 
together with the time stamps. Alternatively, the customer 
information could be stored in the server, in which case it 
would not be transmitted together with the lime stamps. The 
customer information could include a customer code that 
uniquely identifies each subscriber to the system, and 
optionally provides additional information about the sub- 
scriber. If the SDC table is stored in the server, the customer 
number could also be used to access the customer's SDC 
table. 

Once the time stamps and SDC table have been received 
by the server, the time stamps are processed. This processing 
involves additional information, which is preferably stored 
in tables on the server, including the media service table 905 
(shown in FIG. 15) and the broadcast report table 910 
(shown in FIG. 16B). 

The media service table 905 indicates which network 
corresponds to a given channel for a given service. For 
example, the first row of data in the media service table 905 
indicates that channel 20 on service 3888 corresponds to 
USA networks. The second row indicates that channel 14 on 
service 3888 corresponds to HBOl. The service ID field in 
this table indicates the service provider (e.g., Manhattan 
Cable), and corresponds to the service ID field in the SDC 
table 250 (shown in FIG. 4). Each entry on the media service 
table also includes a location code which can be used to store 
information about a time zone. 

A program production table 909 (shown in FIG. 16A) 
holds information that indicates which company produced 
each program, and a unique content ID that identifies the 
program. Some programs may be supported by the server 
one year, but not supported in another year. When only 
supported programs appear in the table 909, the program 
production table can be used to indicate which programs are 
supported by the server. Alternatively, a field can be added 
to each entry in the program production table 909 to indicate 
whether each program is supported. 

The fourth entry in the program production table 909 is an 
example of using the present invention to commercialize 
products that appear within commercials, and it shows how 
commercials can be treated like any other program. For 
example, if the announcer in a commercial happens to be 
wearing a certain hat, and the user presses the event button, 
this event would be processed just like event button presses 
that occur during ordinary programs. Instead of identifying 
a program that corresponds to the time stamp, however, the 
system would identify the commercial that corresponds to 
the time stamp. 

A broadcast report table 910 (shown in FIG. 16B) is 
provided to the server for each individual network supported 
by the server. Alternatively, although not so depicted, a 
single broadcast report table may be used for all networks, 
provided that a network identifier entry is added to the table. 
The broadcast report table 910 is used by the server to 
determine (1) which program the user was watching when 
he pressed the event button; and (2) to provide synchroni- 
zation information which can be used to perform skew 
compensation. 

The first row of data in the illustrated broadcast report 
table 910 indicates that a segment of a program began 
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broadcasting at 8:22 pm and stopped broadcasting at 8:34 
pm, and that the content ID of that program is 2039312. The 
content ID uniquely identifies a particular program (e.g., 
episode #12 of Melrose Place, or an cdited-for-TV version 

5 of The Shawshank Redemption, corresponding to the entry 
in the program production table 909). The synchronization 
information in the two right-hand columns of the broadcast 
report table 910 indicates that at 8:25 pm, the 23rd minute 
of that program was broadcast. 

10 The third row of broadcast report table 910 indicates that 
the same program (i.e., program 2039312) started up again 
at 8:37 pm and continued until 8:45 pm. The synchroniza- 
tion entries for this row coincides with the start of the 
segment, since the fade-in time and the broadcast sync lime 

is are the same (i.e., 8:37 PM, which corresponds to minute 32 
of the program). 

Synchronization entries may be performed semi- 
automatical ly by, for example, having an operator in the 
broadcasting studio enter the synchronization data by noting 

20 that at 8:25 pm, the 23rd minute of a particular show (such 
as episode No. 12 of Melrose Place) was airing. This 
example is illustrated in the first row of data in the broadcast 
report table 910. After the data for a given show (or portion 
thereof) is complete, the broadcast report table entries can be 

25 sent to the server, via, for example, a modem connection. 
More preferably, the synchronization is performed auto- 
matically using a continuous communication link to send 
each entry from a cooperating broadcaster to the server 
individually. For example, each entry may be sent at the start 

30 of each broadcasted segment. In the example illustrated in 
the third row of the broadcast table 910, the broadcaster 
transmits the program synchronization time of 32 minutes at 
8:37 PM, which is the time that they fade in the segment that 
starts at the 32nd minute of the program. When this arrahge- 

35 ment is used, a single entry can be used to represent both the 
fade in lime and ihe broadcast synchronization lime, because 
they will always be the same. 

Optionally, the fade-out time can be eliminated from the 

4Q broadcast report table 910 by using the fadein time from the 
next entry. 

FIG. 17 is a flow chart that shows how the server 
processes each lime slamp to determine the lime within the 
program when ihe event button was pressed, based on Ihe 

4 5 time stamp itself, the SDC table, the media service table, and 
the broadcast report table. This will be explained using the 
example of the second entry in the time stamp table 280 
(shown in FIG. 7). As explained above, copies of this table 
and the SDC table have been sent to the server. 

50 First, in step S1000, the server determines the particular 
service that is associated with the time stamp. To accomplish 
this, the server extracts the luncr device for a given time 
stamp from the time stamp table 280 (shown in FIG. 7). In 
the example under consideration, this would be the cable 

55 box. Then, the server uses this extracted information as an 
index into the service and device configuration table 250 
(shown in FIG. 4) to determine the service that was provid- 
ing the signal to the cable box in the user's home. In the 
example under consideration, this information would be 

60 extracted from the first raw of the service and device 
configuration table, which identifies Manhattan Cable, 
which has an associated service ID code of 3888. In embodi- 
ments where only one station is supported by the system, 
this step can be omitted because the service is always 

65 constant. 

Next, in step S1010, the server determines the network 
that was being watched by the user at the time the event 
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button was pressed. This is accomplished by using the 
service ID determined in step S1000 and the channel 
extracted from the time stamp as indexes into the media 
service table 905 (shown in FIG. 15). In the example under 
consideration, the service ID of 3888 and the channel 5 
appear in the third row of the media service table, which 
indicates that the network is Fox USA. In embodiments 
where only one station is supported by the system, this step 
can also be omitted because the network is always constant. 

Preferably, the media service table also includes a location 
code which indicates the location of the user who pressed the 
event button. This location code is used to compensate for 
users located in different time zones. Optionally, la handle 
cases in which two different networks broadcast over the 
same channel at different times, a time entry (not shown) can 
also be included in the media service table. If this is done, 
the time from the time stamp under consideration would also 
be used as an index into the media service table. 

In step S1020, the server determines the particular pro- 
gram that the user was watching when the event button was 
pressed. This is accomplished by accessing the broadcast 
report table 910 (shown in FIG. 16B) for the particular 
network determined in step S1010, and using the time from 
the time stamp as an index into that broadcast report table 
910. Alternatively, in cases where multiple networks are 
included in a single broadcast report table with an entry to 
specify the network, both the network determined in step 
S1010 and the time from the time stamp are used as indexes 
into the broadcast report table. 

In the example under consideration, the broadcast report 
table 910 is assumed to be the broadcast report table from 
the Fox USA network, and the time extracted from the time 
stamp is 8:26. The server uses this extracted time as an index 
into Fox's broadcast report table 910, and searches for 
entries that correspond to the extracted time. Preferably, this 
is accomplishing by checking if the extracted time is later 
than the fade-in time and earlier than the fade-out time. 
Here, the extracted time of 8:26 falls between the fade-in 
lime and the fade-out lime of the first row of the broadcast 
report table 910. This points to the program with a content 
ID of 2039312, which uniquely identifies the program that 
was being watched when the event button was pressed. 

In cases when the identified program is not supported by 
the system, an appropriate message can be returned to the 
user via the Internet. 

Next, in step S1030, the server performs skew compen- 
sation to determine the particular time within the program 
when the event button was pressed. This is accomplished 
using the synchronization information extracted from the 
broadcast report table 910. The preferred algorithm for 
determining the program time (i.e., the time within the 
program) is implemented using the following equations: 

PTS-PST+BTS-BST 1) 
PSB-PST+FIT-BST 2) 
PSE-PST+FOT-BST 3) 

where PST is the program synchronization time, PSB is the 
program segment begin time, PSE is the program segment 
end time, PTS is the time stamp time measured in program 
time, BTS is the time stamp time measured in broadcast 
time, BST is the broadcast synchronization time, FIT is the 
fade-in lime, and FOT is the fade-oul time. 

When the broadcast report table 910 is provided auto- 
matically at the start of each broadcast segment, as discussed 
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above, the broadcast synchronization time and the fade-in 
time will be the same, which simplifies equation #2. 

Continuing with the example under consideration where 
the broadcast time stamp is 8:26, and plugging the relevant 
5 times into equation 1, the result is: 

PTS-Q:23+S:26-S:25 

Solving this equation results in a program time of 0:24. This 
means that when the event button was pressed at 8:26 PM, 

10 the 24th minute of the program was being displayed. It 
should be noted that while the above example uses one 
minute increments, using smaller increments (e.g., one 
second, or 0.1 seconds) is preferable. 

After determining the lime within the particular program 

is when the event button was pressed, the products that were 
being displayed at that time can be determined by referenc- 
ing the product display table 920 (shown in FIG. 19). 

Preferably, the product display table is created before 
broadcasting the program, and inputted into the server. A 

20 product display table can be made for any given program by 
watching the uncut program and waiting to see whether 
products supported by the system appear. When those prod- 
ucts appear, a product description is entered into the product 
display table, together with an entry for the appear time (i.e. 

25 the time that the product appeared), and a disappear time. Of 
course, alternative data structures may be used, such as 
using a product code instead of the product name, or using 
the duration of appearance instead of the disappear time. 
An example of this process is illustrated in FIG. 18, which 

30 is self-explanatory. An example of the resulting product 
display table 920 is illustrated in FIG. 19. Each time a 
product appears, a new entry is generated in the product 
display table 920. The content ID column of the product 
display table 920 identifies a particular program (e.g., epi- 

35 sode No. 12 of Melrose Place). This process continues until 
the program is finished. The product display table 920 may 
be inputted into the server in any conventional manner. 

Additional information about the supported products are 
also inputted by the server in the product table 930 (shown 

40 in FIG. 20). TTie entries in Ihis lable are self explanatory. 
Any suitable alternative or additional information may also 
be included in the product table 930, including information 
commonly used in computer-assisted marketing systems. 
Returning now to FIG. 17, once the program time has 

45 been determined in step S1030, control passes to step 1040 
where the server searches the product display table for 
entries in which the appear time comes before the time 
stamp at issue, and the disappear time comes after the time 
stamp at issue (allowing for a time tolerance). The server 

50 selects the products that meet these criteria and presents the 
resulting assortment of products to the user. An example of 
this process is illustrated in FIG. 21, which is self- 
explanatory. 

The products can be presented lo Ihe user in batches, with 
55 each batch corresponding to a particular time stamp. 
Optionally, the server may retrieve a still-frame image or 
video clip from the program that recreates the entire image 
that appeared on the user's television of the time when the 
event button was pressed. This image can be transferred to 
60 user's home computer via the Internet in any conventional 
manner to enhance the presentation of the assortment of 
products. 

By implementing this option, the system can easily verify 
that the computed program and time correspond to the 
65 program and lime that was actually being watched by the 
user when the event button was pressed. This would be 
useful, for example, when a user presses a channel -change 
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button on the remote, but the remote is not aimed at the 
television. Id that case, the channel being tracked in the 
remote would not match the channel actually being watched. 
By displaying a still image or video clip from the program, 
the system can verify the computed program and time by 
querying the user. If an error is detected, the system can then 
query the user to determine which program was actually 
being watched, and then reprocess the time stamp for that 
program. 

Alternatively, each product may be presented to the user 
individually, in sequence. As yet another alternative, all of 
the products for all of the time stamps may be presented to 
the user simultaneously. Numerous other presentation 
approaches will be apparent to persons skilled in the relevant 
art. 

Finally, the server continues to operate as a conventional 
web server, using browsing commands from the home 
computer's web browser (which is operated using either the 
remote or the computer keyboard, as described above) to 
provide information and promote the products in the assort- 
ment. 

It is to be understood that the present invention is not 
limited to the specific embodiments described above, and 
that various changes and modifications can be effected 
without departing from the scope or spirit of the present 
invention. One such modification is depicted in FIG. 22 
which shows implementing the present invention in an 
operating system application space on a computer, a TV 
set-top box. or a digital TV. In this configuration, the 
functions of the television, the handheld remote, and the 
home computer are all combined into a single device. 
Similarly, the present invention can also be implemented in 
another devices that supports Internet communications. 
Another possible modification would be to implement the 
system to support only one broadcaster, and simplifying the 
processing performed in the server accordingly. 

Another modification would be to apply the present 
invention to other types of broadcasted video information, 
(including, for example, Internet broadcasts), or even to 
audio broadcasts (including, for example, FM radio). Yet 
another modification would be to send customer specified 
information from the computer to the remote during data 
transfer sessions. This information could include, for 
example, birthday and anniversary reminders that are ini- 
tially entered at the home computer. At the appropriate time 
(e.g., five days before the birthday), the remote can be 
programmed to display an appropriate message on the LCD 
display. Slill another modification would be to distribute the 
various functions among different locations, by, for 
example, having the broadcaster perform some of the server 
processes described above. Numerous other alternative 
embodiments can be readily envisioned. 

We claim: 

1. A method of commercializing products that are present 
in a television broadcast of a program, the method compris- 
ing the steps of: 

inputting product information that identifies a plurality of 
products which are present in the program and a time 
of presence within the program for each of the prod- 
ucts; 

inputting skew information that identifies a correspon- 
dence between an actual time of broadcast and a 
relative time within the program for at least one seg- 
ment of the program; 

inputting a time marker that identifies a time selected by 
a user while watching the television broadcast; 

identifying a specific portion of the program that was 
being broadcast at the time selected by the user based 
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on the time identified by the time marker and the 
inputted skew information; 
determining an assortment of products that were present 
in the television broadcast at the time selected by the 
5 user based on the specific portion identified in the 
identifying step and the inputted product information; 
and 

presenting the determined assortment of products to the 
user. 

10 2. The method of claim 1, wherein the television broad- 
cast includes a plurality of programs that are broadcasted in 
turn, and the step of identifying a specific portion of the 
program comprises the steps of: 

identifying one program selected from the plurality of 
15 programs based on the time identified by the time 
marker and inputted broadcast information that identi- 
fies when the one program was broadcasted; and 
identifying a specific portion of the program that was 
being broadcast at the time selected by the user based 
on the time identified by the time marker and the 
20 inputted skew information. 

3. The method of claim 1, further comprising the steps of: 
receiving an indication from the user selecting a product 

from the presented assortment; and 
providing, to the user, information relating to the selected 
25 product. 

4. The method of claim 3, wherein said step of providing 
information to the user comprises at least one of: offering the 
selected product for sale, providing information about the 
selected product, providing a referral to a seller of the 

30 selected product, and providing a link to a seller of the 
selected product. 

5. The method of claim 1, wherein the product informa- 
tion includes, for each of the products, an appear time and 
a disappear time within the program. 

35 6. The method of claim 1, wherein the skew information 
is inputted from a television station during the television 
broadcast of the program. 

7. The method of claim 1, wherein the product informa- 
tion is inputted from a database prior to the television 

40 broadcast of the program. 

8. The method of claim I, wherein, in said time marker 
inputting step, the time marker is inputted via the Internet, 
and 

wherein, in said presenting step, the determined assort- 
45 ment of products are presented to the user via the 
Internet. 

9. The method of claim 1, wherein said step of presenting 
the determined assortment of products to the user comprises 
transmitting to the user, via the Internet, an image of a scene 

50 that was being broadcast at the time selected by the user. 

10. A method of commercializing products that are 
present in a plurality of simultaneous television broadcasts 
of a plurality of programs over a plurality of channels, the 
method comprising the steps of: 

55 inputting, for each of the programs respectively, product 
information that identifies a plurality of products which 
arc present in the respective program and a lime of 
presence within the respective program for each of the 
products; 

60 inputting, for each of the programs respectively, skew 
information that identifies a correspondence between 
an actual time of broadcast and a relative lime within 
the respective program for at least one segment of the 
respective program; 

65 inputting a marker that identifies a time selected by a user 
while watching any of the television broadcasts and a 
channel being watched by the user; 
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identifying a specific portion of a specific program thai 
was being watched by the user at the time selected by 
the user based on time and channel identified by the 
marker and the inputted skew information; 

determining an assortment of products that were present 5 
in the specific program at the time selected by the user 
based on the specific portion identified in the identify- 
ing step and the inputted product information; and 

presenting the determined assortment of products to the 
user. 30 

11. The method of claim 10, wherein each of the televi- 
sion broadcasts includes a plurality of programs that are 
broadcasted in lum, and the step of identifying a specific 
portion of the program comprises the steps of: 

identifying a broadcaster based on the channel identified 
by the marker; 

identifying one program selected from the plurality of 
programs for the identified broadcaster, based on the 
time identified by the marker and inputted broadcast 2 o 
information, wherein the inputted broadcast informa- 
tion identifies when the one program was broadcasted 
by the identified broadcaster; and 

identifying a specific portion of the program that was 
being broadcast at the time selected by the user based 25 
on the lime identified by the marker and the inputted 
skew information. 

12. The method of claim 10, further comprising the steps 

of: 

receiving an indication from the user selecting a product 30 

from the presented assortment; and 
providing, to the user, information relating to the selected 

product. 

13. The method of claim 12, wherein said step of pro- 
viding information to the user comprises at least one of: 35 
offering the selected product for sale, providing information 
about the selected product, providing a referral to a seller of 
the selected product, and providing a link to a seller of the 
selected product. 
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14. The method of claim 10, wherein the product infor- 
mation includes, for each of the products, an appear time and 
a disappear time within the program. 

15. The method of claim 10, wherein the skew informa- 
tion is inputted from respective television stations during 
respective television broadcasts of respective programs. 

16. The method of claim 10, wherein the product infor- 
mation is inputted from a database prior to the television 
broadcast of each program. 

17. The method of claim 10, wherein, in said marker 
inputting step, the marker is inputted via the Internet, and 

wherein, in said presenting step, the determined assort- 
ment of products are presented to the user via the 
Internet. 

18. The method of claim 10, wherein said step of pre- 
senting the determined assortment of products to the user 
comprises transmitting to the user, via the Internet, an image 
of a scene that was being watched by the user at the time 
selected by the user. 

19. A method of commercializing products that are 
present in a plurality of simultaneous television broadcasts 
of a plurality of programs over a plurality of channels, the 
method comprising the steps of: 

inputting, for each of the programs respectively, broadcast 
and product information that identifies a plurality of 
products which arc present in the respective broadcast 
and a time of presence within the respective broadcast 
for each of the products; 

inputting a marker that identifies a time selected by a user 
while watching any of the television broadcasts and a 
channel being watched by the user; 

determining an assortment of products that were present 
in a specific program that was being watched by the 
user at the time selected by the user based on the time 
and channel identified in the marker and the inputted 
broadcast and product information; and 

presenting the determined assortment of products to the 
user. 

* * * * * 
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